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Executive  Summaiy 


Approximately  20%  of  households  provided  a  written  comment  on  their  NCT 
questionnaire.  The  distribution  of  these  written  comments  was  somewhat  dispersed 
with  a  majority  of  these  households  (67%)  providing  a  single  written  comment  and 
a  fiirthCT  20%  providing  two  written  comments. 

The  writtoi  comments  were  highly  concentrated  on  specific  NCT  questions.  Income, 
ethnic  origin  and  unpaid  work  questions  were  the  most  frequentiy  mentioned  NCT 
questions  in  terms  of  both  difficulty  and  objections. 

Less  than  20%  of  responding  households  indicated  that  they  had  consulted  the  Guide 
incompleting  their  NCT  questionnaire.  While  just  over  4%  of  responding  households 
reported  that  they  had  used  the  Census  Help  Line  (CHL),  an  overwhelming  majority 
(84%)  foimd  this  service  helpful.  Households  completing  a  French  questionnaire  were 
less  lil^y  to  find  the  Guide  and  the  CHL  of  assistance. 

An  examination  of  the  writtra  comments  reveals  two  different  types  of  responding 
households.  The  first  type  could  be  said  to  be  generally  mistrustftil  of  government 
and,  as  such,  tended  to  find  the  NCT  unnecessary  and  many  questions  intrusive.  The 
second  type  of  responding  household  provided  comments  which  tended  to  reflect  a 
more  sincere  concern  with  a  specific  NCT  question. 

The  majority  of  written  comments  (56%)  can  be  categorized  as  "response  burden" 
while  28%  of  comments  can  be  classified  as  "privacy  concerns"  or  "other  objections". 
However,  the  written  comments  in  the  aggregate  seem  to  indicate  that  Statistics 
Canada  is  approaching  the  limits  of  accq)tability  in  terms  of  questionnaire  l^gth  and 
complexity. 

It  is  within  the  context  of  specific  NCT  content  that  Step  10  is  of  most  value.  That 
is,  while  this  report  provides  an  "accounting"  of  flie  responses  and  a  g^ieral  overview 
of  written  comments,  it  is  not  exhaustive  and  cannot  fully  convey  the  reaction  of 
households  to  specific  subject  matter  concepts  and  questions. 
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Environ  20  %  des  m&iages  ont  insciit  un  commentaire  sur  leur  questionnaire  du 
TRN,  La  r6partition  des  commentaires  Merits  6tait  quelque  peu  dispersde,  la  majority 
des  manages  (67  %)  ayant  foumi  un  seul  commentaire  6cnt  et  un  autre  20  %  ayant 
foumi  deux  commentaires  Merits. 

Une    proportion    61ev6e   des   commaitaires    6crits   portaient    sur  des   questions 
spdcifiques  du  TRN.    Par  exemple,  les  questions  du  TRN  siir  le  revenu,  I'origine 
ethnique    et  le  travail   non   T6mun6r6    sont  celles  qui  ont  suscit6  le  plus  de 
commaitaires  tant  k  cause  des  difficult^  que  des  objections  qu'elles  soulevaient. 

Moins  de  20  %  des  m6nages-F^ndants  ont  indiqu6  avoir  consult6  le  Guide  au 
moment  de  remplir  leur  questionnaire  du  TRN.  Bi«i  que  juste  un  peu  plus  de  4  % 
des  m^nages-rdpondants  aient  indiqu6  avoir  eu  recours  k  I'Assistance  t^l^honique 
du  recensement  (ATR),  une  majority  surprenante,  soit  84  %,  a  reconnu  rutilit6  de 
ce  service.  Les  manages  k  qui  on  avait  remis  un  questionnaire  firangais  6taient  moins 
susceptibles  de  trouver  une  utility  au  Guide  et  k  I'ATR. 

Un  examen  des  commentaires  &rits  fait  ressortir  deux  diffdrents  types  de  m6nages- 
r^ndants.  Du  premier  type  de  m6nage-r6pondant,  on  pent  dire  en  g6n6ral  qu'il 
6tait  m6fLant  k  regard  du  gouvemement  et  qu'il  avait  tendance  de  ce  fait  k  trouver 
le  TRN  inutile  et  k  trouver  bon  nombre  des  questicms  indiscr^tes.  Le  m6nage- 
rSpondant  du  deuxieme  type  foumissait  des  commentaires  qui  tendai^it  k  d6montrer 
une  inquietude  veritable  fece  a  une  question  sp&ifique  du  TRN. 

La  majorit6  des  comm^talres  6mts,  soit  56  %, peuvoit  6tre  classes  dans  la  cat6gorie 
«£ardeau  du  r6pondant»,  tandis  que  28  %  des  comm^taires  se  qualifient  comme 
6tant  des  preoccupations  k  regard  de  la  *vie  priv€e»  ou  d'«autres  objections*. 
Toutefois,  dans  I'ensemble,  les  commrataires  dciits  sembl^t  indiquer  que  Statistique 
Canada  n'est  pas  loin  d'atteindre  les  limites  de  ce  que  Ton  con^d^  accq)table  pour 
un  questionnaire  en  termes  de  longueur  et  de  complexit6. 

C'est  dans  le  contexte  du  contenu  specifique  du  TRN  que  I'^tape  10  s'avfere  le  plus 
utile.  Cela  veut  dire  que  mSme  si  le  present  rapport  Mt  la  «comptabilit6»  des 
r^ponses  et  donne  un  apergu  general  des  comm^itaires  6crits,  il  n'est  toutefois  pas 
complet  et  ne  peut  6voquer  pleinement  les  inactions  des  m^ages  face  k  des 
questions  et  k  des  concepts  specialises  bien  precis. 
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1.      Prqiatatioii  for  the  1996  C^isus 

In  preparing  for  the  1996  Census  of  Population,  Statistics  Canada  conducted  a  test  of  the 
content  and  of  selected  field  procedures  and  support  activities  in  November  of  1993.  The 
National  Census  Test  (NCT)  was  based  on  a  sample  comprised  of  two  components.  A 
rq>resentative  sample  of  approximately  14,500  occupied  dwellings  was  selected  from 
"rotations  out"  of  the  Labour  Force  Survey  (LBS).  A  special  population  sample  of 
approximately  3,500  occupied  dwellings  was  designed  to  target  specific  ethno-cultural 
communities  in  selected  uri}an  areas.  The  NCT  represents  the  sole  opportunity  to  test  new 
or  modified  questions  and  procedures  (i.e.  automated  coding  modules)  for  the  1996  Census. 

The  NCT  questionnaire  included  a  section  (Step  10)  containing  questions  designed 
specifically  to  solicit  the  reaction  of  respondents  to  the  test  questionnaire  (Appendix  A).  This 
report  is  an  evaluation  of  the  answers  and  conunents  provided  by  households  to  these  Step 
10  questions.  The  first  section  of  this  report  is  a  quantitative  analysis  of  the  answers  to  Step 
10  questions  while  the  second  section  is  a  qualitative  examination  of  the  actual  written 
commits  provided  to  these  questions.  This  report  is  intended  to  identify  problems  with  the 
questionnaire  (e.g.  design,  wording,  concepts)  for  subject  matter  consideration  and  to  alert 
the  1996  Census  collection  and  communications  projects  as  to  potential  "trouble  spots". 


2.      Respondoit  Evaluation  of  the  NCT 

2.1Approadhi 

The  responses  to  Stq)  10  were  processed  as  part  of  the  hous^old  file  and,  since  the  LFS 
and  Special  components  of  the  NCT  sample  were  split  into  separate  files,  the  results  for 
each  are  presented  separately  in  this  reporL  Responding  private  housdiolds  in  occupied 
dwellings  generated  a  final  sample  of  12,273  households  for  the  LFS  componait  and  2,839 
housdiolds  for  the  Special  componoit.*  There  are  seveal  caveats  which  must  be  considered 
in  reviewing  the  numb^s  in  this  report.  First,  the  tables  are  based  on  the  unweighted  or 
"raw"  counts  since  it  was  decided  that  providing  wdghted  counts  would  serve  no  purpose 
and,  moreover,  weights  are  not  available  for  the  Special  sample  compon^t.   It  is  also 
important  to  note  that  these  files  have  not  been  edited  and  inconsistencies  of  respondents 
have  been  captured  and  processed  "as  is".  An  investigation  of  the  files  reveals  that  missing 
values,  rather   than  inconsistrat   responses,    seem  to  be  the  more  significant  problem 
(Appradix  B).  As  a  final  cautionary  note,  NCT  Step  10  questions  were  excluded  fi'om  field 
edits  ^.e.  "Rule  of  Six")  and,  for  sdl  practical  purposes,  these  questions  were  not  included 
in  the  interviewCT  follow-up  procedures. 


^  Unix  files  "nct.lftam|).hhldfile.dat.01.q)r21"(LFS)  and  "nctspecpop.lihldfile.dat.01.apr.21"  (Special). 
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The  quantitative  analysis  consists  of  six  tables  corresponding  to  Step  10  questions  A  to  F  on 
the  NCT  questionnaire-  Each  table  reports  the  sample  size  and  the  household  answers  to 
the  question.  The  "No  Response"  value  rqwrted  is  calculated  residually  since  the  total  of 
missing  values  for  each  field  will  exceed  the  sample  size.^  A  selective  frequency  of  step  or 
question  numbers  is  also  included  in  these  tables.  An  exhaustive  frequency  of  step  or 
question  numbers  for  Step  10  is  contained  in  a  precursory  document  to  this  report  (NCT 
RqKMt  25  -  Preliminary  Analysis).  Percentages  in  each  table  are  calculated  only  for  those 
households  which  responded;  missing  values  are  excluded.  Moreover,  these  tables  provide 
selective  breakdowns  of  the  answers  by  Regional  Office,  Formtype  (English/French),  Person 
Number  (who  completed  the  questionnaire)  and  Household  Size.  A  frequency  of  these 
breakdown  variables  is  contained  in  Appendix  C.  Explanatory  notes  accompany  each  table 
to  point  out  particular  nuances  regarding  the  values. 

Since  no  editing  of  the  responses  was  undertaken,  the  issue  of  a  response  rate  is  not 
straightforward.  As  such,  this  report  wiU  use  an  incidaice  rate  of  Stq)  10  comments.  The 
incidence  rate  is  based  on  the  housdhold  responding  to  at  least  one  of  Step  10  questions  A, 
B,  D  or  F,  indicating  a  stqp  or  question  number  to  this  response,  and  providing  a  written 
conraient.^  Of  the  15,112  households  on  the  two  data  files,  2,915  or  19.3%  of  the 
questionnaires  satisfied  this  condition.  Ttiis  breaks  down  into  2,471  or  20.1%  of  households 
in  the  LPS  sample  component  and  444  or  15.6.  %  of  the  Special  sample  component.  These 
incidence  rates  should  be  considered  a  conservative  estimate  at  the  lower  end  of  the  possible 
range  of  response  rates.  For  botii  the  LFS  and  Spedal  sample  compon^ts,  only  75%  of 
households  which  indicated  a  step  or  question  number  in  Step  10  (Questions  A,  B,  D  and 
F)  also  provided  a  reason  or  explanation  in  the  form  of  a  written  comment. 

These  2,915  "responding  households"  which  provided  at  least  one  wiittai  comment  in  Step 
10  together  generated  a  total  of  4,526  comments  (LFS  3,820  +  Special  706)  for  an  average 
of  1.6  commaits  per  household.  Again,  these  rates  are  based  on  the  concUtion  that  a  stq> 
or  question  numba-  is  provided  gnd  the  flag  indicating  a  writt^  comment  is  present.  TTie 
Special  sample  componait  averaged  slightly  fewer  written  commits  at  1.55  per  household. 
Appro3dmately  67%  of  the  households  wWch  responded  to  Step  10,  as  measured  by  the 
incidence  rate,  provided  a  single  written  comment  while  about  20%  provided  two  comments. 
Only  about  12%  of  responding  households  provided  three  or  more  comments. 

2.2  Analysis 

To  rdterate  before  examining  specific  questions  in  Stsp  10,  about  one  in  five  housdiolds 
provided  at  least  one  written  comment  and  a  majority  of  these  households  (>  65%)  provided 
a  single  writtwi  comment.  The  tables  which  follow  preset  selective  results  from  each  of  the 


^  For  exanqile  in  Question  A,  the  LFS  "No  Response"  value  of  2,251  is  calculated  by  subtracting  the 
STEP10A1=I  (9.732)  and  the  STEP10A2=2  (290)  sub-totals  from  the  sample  size  (12,273). 

^  The  writt^i  comments,  while  not  data  captured,  are  flagged  by  the  presence  of  a  1  in  a  subsequent  field. 
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six  NCT  Step  10  questions. 


Table  1 


Question  A:  Did  you  find  any  of  the  st^s 

on  pages  2  and  3  difficult?                             || 

LPS 

Spedal 

Households  (n) 

12,273 

2,839 

Yes 

290 

68 

No 

9,732 

2,077 

1    No  Response 

2,251 

694 

Yes  /  (Yes  +  No) 

2.9% 

3.2% 

Edmonton  R.O. 

0.8% 

3.0% 

Monti^  R.O. 

3.7% 

5.2% 

Total  Steps  [1] 

194 

37 

Stq)  4  [2] 

m 

8 

Step  2 

47 

16 

Other  Steps 

87 

13 

[1]    Housdiolds  could  mark  up  to  5  step  numbers. 

[2]  NCT  Rqport  11  (Coverage-Related  Steps  and  Stq)  9)  reports  tiiat,  out  of  the  68 
cases  of  housdiolds  indicating  difficulty  with  Stq>  4,  there  were  only  S  valid  responses 
and  63  cases  of  error.  Most  of  this  error  stemmed  from  households  inconectiy 
indicating  Question  46  in  Question  A  rather  than  C^estion  B.  The  "4"  occurs  since 
only  one  byte  could  be  captured  for  these  fields  in  Question  A. 

The  reported  difficulties  with  NCT  step  numbers  ranged  fix)m  less  than  1%  of  LPS 
housdiolds  for  the  Edmonton  Regional  Office  to  over  5%  of  Special  household  for  the 
Montreal  Regional  Office.  Step  2  provided  the  most  difficulties  with  routine  problems 
regarding  the  ordering  of  names  and  the  inclusion  of  students  (NCT  Report  11). 

While  just  3%  of  households  reported  difficulties  with  the  NCT  steps,  closer  to  14%  of 
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households  indicated  difficulty  with  at  least  one  NCT  question  (Table  2).  Not  unexpectedly, 
Table  2  indicates  that  a  larger  household  of  more  than  6  persons,  requiring  two 
questionnaires,  would  be  more  likely  to  report  difficulties  with  particular  NCT  questions  than 
would  a  household  of  1  person.  The  five  most  frequentiy  mentioned  NCT  questions  are  also 
contained  in  Table  2.  It  is  assumed  that  the  difficulties  households  reported  for  specific  NCT 
questions  are  addressed  in  more  detail  by  the  subject  matter  areas. 


Table  2 


Question  B:  IH^d  you  find  any  of  the  test 
questions  difticdt? 

LFS 

Spedal 

Households  (n) 

12,273 

2,839 

Yes 

1,325 

275 

No 

8,159 

1,774 

No  Response 

2,789 

790 

Yes  /  (Yes  +  No) 

14.0% 

13.4% 

Housdiold  Size  =  1 

12.2% 

8.6% 

Household  Size  >  6 

22.0% 

6.7% 

Total  Questions  [1] 

1,851 

374 

1      Q46  (Income) 

814 

152   1 

Q16  (Ethnic  Origin) 

125 

17 

Q26/27  (Unpaid  Work) 

105 

21 

Q44  (Weeks  Worked) 

88 

19 

Q34  (Name  of  Employer) 

69 

17 

Other  Questions 

650 

148 

[1]     Households  could  mark  up  to  5  question  numbers. 
Table  3  contains  the  number  of  households  which  reported  that  they  had  consulted  the  NCT 


-6- 


guide  to  complete  their  questionnaires.  Approximately  18.2%  of  responding  households 
reported  that  they  had  consulted  the  guide.  This  underscores  the  perception  that  the  guide 
in  its  current  form  is  not  effectively  utilized.  The  Table  also  reveals  that  in  completing  the 
NCT  questionnaire,  Person  1  was  much  more  likely  to  consult  the  guide  than  was  a  person 
not  from  the  household.  With  respect  to  the  reason  the  guide  was  consulted,  it  should  be 
noted  that  there  were  no  instructions  in  the  NCT  Guide  explicitly  for  dther  Step  8  or  Step 
9.  It  ^jpears  that  households  indicated  the  Step  number  containing  the  question(s)  for  which 
the  Guide  was  consulted. 


Table  3 


Question  C:  Did  you  use  the  guide? 

LPS 

Spedal 

Sample  (n) 

12,273 

2,839 

Yes 

1,888 

301 

No 

8,030 

1,822 

No  Response 

2,355 

716 

1     Yes  /  (Yes  +  No) 

19.0% 

14.2%    1 

Person  1 

19.9% 

15.1% 

"Other"  Person  [1] 

8.9% 

5.6% 

Total  Steps/Questions  [2] 

2,972 

488 

Steps  8/9 

765 

116 

Q46  (Income) 

492 

26 

out-of-scope  [3] 

232 

71 

Q16  (Ethnic  Origin) 

160 

19 

Q35  (Kind  of  Business) 

100 

12 

1      Other  Steps/Questions 

1,223 

244 

[1]     Person  who  completed  the  questionnaire  who  does  not  reside  in  household. 
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[2]     Households  could  mark  up  to  6  step  numbers  and  6  question  numbers. 

[3]  The  out-of-scope  figures  for  Question  C  appear  to  be  the  result  of  households  not 
correctly  marking  in  the  question.  For  example,  there  are  122  instances  in  the  LFS 
file  of  households  writing  in  Question  number  46  (Income)  in  the  Step  No.  mark-in. 

For  those  households  which  consulted  the  guide,  about  two-thirds  indicated  that  it  was 
helpful  (Table  4).  However,  those  households  responding  using  a  French  questionnaire  were 
less  likely  to  find  the  Guide  helpful.  For  the  step/question  numbers  reported  by  households 
for  which  the  guide  was  not  helpfiil,  out-of-scope  and  Steps  8/9  were  the  most  frequent. 
Again,  it  appears  that  households  indicated  the  Step  number  containing  the  question(s)  for 
which  the  Guide  was  consulted. 


Table  4 


Question  D:  Was  the  guide  hdpfiil? 

LFS 

Special 

1     Households  [1] 

3,444 

646 

Yes 

2,313 

432 

No 

1,131 

214 

No  Response 

n/a 

n/a 

No  /  (Yes  +  No) 

32.8% 

33.1% 

French  [2] 

54.6% 

63.1% 

English  [2] 

23.4% 

23.3% 

Total  Steps/Questions  [3] 

279 

48 

1       out-of-scope    [4] 

66 

17    II 

Steps  8/9 

62 

12 

Other  Steps/Questions 

151 

19 

[1]     It  is  not  possible  to  differentiate  between  households  which  did  not  answer  Question 
D  (i.e.  missing  values)  and  those  for  which  Question  D  does  not  apply. 
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[2]     Statistically  significant  difference  (os  =  .01),  see  Appendix  D  for  details. 
[3]     Households  could  mark  up  to  6  step  numbers  and  6  question  numbers. 
[4]     These  out-of-scope  counts  result  from  households  incorrectly  providing  a  question 
number  in  a  step  number  mark-in. 

A  comparison  of  Table  3  with  Table  4  indicates  that  more  households  claimed  the  guide 
helpful  (2,745)  than  claimed  to  have  actually  used  the  guide  (2,189).  This  overt  discrq)ancy 
is  the  result  of  households  not  correctly  following  the  skip  pattern  in  Step  10  Question  C. 
An  investigation  reveals  that  approximately  20%  of  housdiolds  which  answered  no  to 
Question  C  missed  the  skip.  That  is,  although  these  households  answered  in  Question  C  that 
they  did  not  use  the  NCT  Guide,  these  same  households  then  subsequently  went  on  to 
comment  on  the  helpfulness  of  the  Guide  in  Question  D  (Appendix  B).  This  certainly 
underscores  tiie  findings  of  NCT  Report  5  (Interviewer  Debriefing)  and  NCT  R^ort  9 
(Evaluation  of  Design  and  Layout)  that  respondents  have  difficulty  following  skip  patterns 
in  the  questionnaire. 


Table  5 


-    ., . 

Question  E:  Did  you  use  the  Craisus  Hdp  line? 

LPS 

Special 

1    Sample 

12,273 

2,839 

1     Yes 

375 

131 

No 

9,490 

1,959 

No  Response 

2,408 

749 

Yes  /  (Yes  +  No) 

3.8% 

6.3%   1 

Was  it  helpful?                                    | 

Sub-Sample 

375 

131 

Yes 

292 

112 

No 

62 

14 

Yes  /  (Yes  +  No) 

82.5% 

88.9% 

EngUsh  [1] 

84.1% 

88.8% 

French  [1] 

73.2% 

90.0% 
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[1]     Statistically  significant  difference  (a  =  .05),  see  Appendix  D  for  details. 

In  Step  10  Question  E,  just  over  4%  of  responding  households  indicated  that  they  had  used 
the  Census  Help  Line  (CHL)  and  approximately  84%  of  these  households  found  the  CHL 
to  be  helpful  (Table  5).  These  figures  also  revKil  that  households  from  the  Special  sample 
component  were  more  likely  to  use  the  CHL.  Again,  those  housdiolds  using  a  French 
questionnaire  were  less  likely  to  find  the  CHL  hdpful.  NCT  Kqxut  4  (Evaluation  of  CHL) 
reports  that  634  calls  were  answered  by  the  Census  Help  Line  operators.  According  to  NCT 
Step  10  Question  E,  506  households  (375  LPS  +131  Special)  indicated  that  they  had  used 
the  CHL.  Assuming  that  these  two  numbers  are  accurate,  this  means  that,  on  average,  each 
household  using  the  CHL  made  1.2  telephone  calls.  However,  tiiis  is  probably  inflated  by 
households  which  did  in  feet  use  the  CHL  but  did  not  complete  Question  E  of  St^  10. 


Table  6 


Question  F:  Are  tii^e  any  test  questions  to 
1                             whidi  you  have  objections?                              |{ 

LPS 

Spedal       1 

Sample  (n) 

12,273 

2,839 

Yes 

2,197 

402 

No 

6,715 

1,540 

1    No  Response 

3,361 

897 

Yes  /  (Yes  +  No) 

24.7% 

20.7% 

Total  Questions  [1] 

2,824 

638 

1      Question  46  (Income) 

2,000 

429 

1      Question  16  (Ethnic  Origin) 

100 

20 

Question  26/27  (Unpaid  Work) 

98 

22 

1      Question  18  (Pop  Group) 

87 

36 

Question  42  (Place  of  Work) 

78 

21 

Other  Questions 

461 

110    1 
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[1]     Households  could  mark  up  to  6  questions. 

The  final  question  of  Step  10  asked  households  if  there  were  any  test  questions  to  which 
they  had  objections.  Approximately  24%  of  responding  households  indicated  that  they  had 
objections  and  there  was  no  significant  variation  in  this  proportion  by  the  breakdown 
variables.  Table  6  contains  the  most  frequently  m^itioned  NCT  questions  to  which 
households  indicated  an  objection.  The  sequ^icing  of  Questions  46, 16  and  26/27  in  Table 
6  is  identical  to  that  of  Table  2  (question  difficulty).  However,  it  is  important  to  note  that 
Question  18  (Population  Group)  is  a  question  households  find  objectionable  although  they 
do  not  find  it  overly  difficult.  With  Question  42  (Place  of  Work),  households  find  it 
somewhat  difficult  (i.e.  postal  code)  and  somewhat  objectionable.  An  examination  of  written 
comments  suggests  that  objections  to  Question  42  stem  from  a  misunderstanding  of  the  use 
of  these  data.  Again,  it  is  assumed  that  individual  subject  matter  areas  will  address  these 
concerns  in  more  detail. 

2.3  Synopsis 

Thore  are  relatively  few  written  comments  in  Step  10  with  only  about  every  5th  household 
(19.3%)  on  average  providing  a  writtMi  comment.  However,  these  questions  added  to 
response  fetigue  given  the  sheer  number  of  NCT  questions  and,  in  addition,  the  placement 
of  Step  10  questions  and  their  exclusion  from  field  follow-up  procedures  would  also 
contribute  to  this  relatively  low  incidence  rate.  Of  the  households  that  did  provide  at  least 
one  written  comment,  a  vast  majority  (67%)  provided  a  single  written  commoit.  This  would 
characterize  the  distribution  of  Step  10  comment-providing  housdiolds  as  somewhat 
dispersed.  However,  while  the  written  comments  are  somewhat  dispersed  among  the 
households,  they  tend  to  be  highly  concentrated  on  specific  NCT  questions. 

It  is  important  to  note  that  those  housdiolds  responding  using  an  English  questionnaire  were 
more  litely  to  find  the  NCT  Guide  and  the  Census  Help  Line  helpful.  Although  these 
differences  are  based  on  smaller  sample  sizes  because  they  are  conditional  responses,  they 
are  statistically  significant  and  therefore  require  further  attention.  On  balance  there  does  not 
appear  to  be  significant  variation  between  the  LFS  and  Special  sample  compon^ts  at  the 
aggregate  level.  This  consistency,  however,  taids  to  disappear  as  the  analysis  gets  more 
detailed  or  focused  on  specific  NCT  questions.  At  the  level  of  evaluating  specific  NCT 
questions,  Stq)  10  is  particularly  germane  in  pointing  out  to  subject  matter  areas  why  a 
question  was  found  by  respondents  to  be  problematic.  The  full  value  of  Step  10  is  not 
exploited  in  this  report  since  the  analysis  remains  at  a  more  general  level. 
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3.      Re^mnd^it   Comments  on  the  NCT 

3.1  SdLection 

The  previous  section  represents  a  quantitative  analysis  of  answers  to  Step  10.  This  section 
presents  a  qualitative  examination  of  the  written  comments  provided  by  households.  As 
previously  maitioned,  the  written  comments  were  not  data  captured  but  rather  flagged  on 
the  final  data  files.  It  was  decided  that  an  examination  of  every  NCT  questionnaire 
containing  a  written  comment  was  not  feasible  and  a  strategy  was  designed  to  select  a 
sample. 

The  sampling  frame  was  established  by  identifying  all  NCT  questionnaires  for  which  a  valid 
entry  was  captured  in  the  first  field  ^,e.  mark-in  box)  of  Step  10  Questions  A,  B,  D  and  F.^ 
This  condition  deliberately  excluded  Question  46  since  it  accounted  for  a  very  high 
proportion  of  all  comments.  In  essence,  a  stratified  sample  was  created  with  a  Question  46 
stratum  and  an  "all  other  questions"  stratum.  The  Question  46  stratum  did  not  contribute 
questionnaires  to  the  sample.^  This  exclusion  of  Question  46  firom  the  condition  did  not, 
however,  eliminate  all  Question  46  comments  since  questionnaires  are  included  for  which 
Question  46  is  found  beyond  the  first  field  or  mark-in  box  of  Step  10  Questions  B,  D  and 
F.  In  fact,  Question  46  accounts  for  116  of  the  616  question-specific  commente  examined. 

Using  this  condition,  there  were  a  total  of  1,286  NCT  questionnaires  identified.  It  was 
decided  to  selected  20%  of  these  questionnaires  for  examination  and  to  proceed  by 
examining  every  5th  questionnaire.  This  resulted  in  a  stratified,  systematic  sample  of  258 
NCT  questionnaires  or  approximately  9%  of  the  total  NCT  questionnaires  with  at  least  one 
written  comm^it  ^.e.  the  sampling  frame).  These  comments  were  examined  in  the  Census 
Mcrofilm  area  with  direct  data  entry  into  a  micro-computer  using  a  pre-determined  coding 
and  classification  scheme  (Appendix  £). 

3.2  Examiiialion 

The  258  NCT  questionnaires  examined  generated  a  total  of  718  written  comments  for  an 
average  of  2.78  written  comments  for  responding  households.  The  210  LFS  households 
averaged  2.83  written  commits  compared  to  2.54  written  comments  for  the  48  households 
in  the  Special  sample  component.  These  averages  are  higher  than  the  1.6  written  comments 


^  In  the  case  of  Step  10  Questioa  D,  the  eatiy  for  the  first  St^  No.  box  coiild  be  from  1  to  49  in 
recc^nition  Uiat  many  respondmts  used  the  first  available  oppoituni^  to  write  the  question  number  of  >^di 
they  had  concerns,  difficulties  or  objections. 

•^  As  an  exunple,  for  the  first  field  of  Step  10  Question  B  the  condition  was  as  follows:  IF 
0<STP10B1Q<46  OR  46<STP10B1Q<50THEN  DO... 
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per  responding  household  reported  in  Section  2. 1  of  this  report.  The  difference  is  accounted 
for  by  two  factors.  First,  the  qualitative  examination  included  written  comments  contained 
on  the  back  cover  of  the  NCT  questionnaires  and  these  comments  were  not  data  captured. 
Second,  it  also  depends  on  how  a  household  answered  Step  10  questions.  If  a  responding 
household  felt  that  a  particiilar  series  of  questions  were  difficult  (e.g.  Questions  28  to  33), 
it  might  be  indicated  by  separately  writing  each  question  number  in  the  six  mark-in  boxes 
of  Step  10  Question  B  or  the  household  may  indicate  this  range  by  writing  28-30  in  the  first 
mark-in  box.  In  the  latter  case,  only  Question  28  would  be  data  captured.  During  the 
qualitative  examination  of  NCT  Questionnaires,  all  six  questions  would  be  captured  as 
separate  comments,  thus  resulting  in  the  higher  number  reported. 

A  summary  of  the  written  comments  examined  is  contained  in  Table  7.  As  expected.  Step 
10  Questions  B  and  F  account  for  approximately  80%  of  the  written  comments  and  most  of 
the  written  comments  concerned  specific  NCT  questions.  A  majority  of  the  written 
comments  examined  can  be  categorized  as  resulting  from  "Response  Burden"  due  to  the 
difficulty,  length  or  perceived  irrelevance  of  particular  NCT  questions.  Approximately  28% 
of  the  written  comments  can  be  categorized  as  resulting  from  "Privacy  Concerns  or  Other 
Objections".  Most  written  comments  tended  to  be  of  a  negative  nature;  just  over  6%  of  the 
written  comments  can  be  classified  as  a  content  or  format  suggestion.  This  should  not  be 
surprising  since  the  Step  10  questions  were  designed  to  solicit  responses  in  cases  which  a 
household  found  a  step  or  question  to  be  difficult  or  objectionable. 


Table  7 


Distdbution  of  Written  Comments 

/ 

« 

Wnttea  Comments 

718 

Step  10  Question  A 

21 

2.92 

Step  10  Question  B 

298 

41.50 

Step  10  Question  D 

60 

8.36 

Step  10  Question  F 

272 

37.88 

Questionnaire  back  cover 

67 

9.33 

NCT  Steps 

33 

4.60 

NCT  Questions 

616 

85.79 

Not  Step/Question  Specific 

69 

9.61 
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Dislributioa  of  Writtoi  Conun^its 

/ 

======= 

% 

Response  Burden 

391 

54.46 

Privacy  Concern 

58 

8.08 

Non-Privacy  Objection 

150 

20.89 

1     Content/Fonnat   Suggestion 

48 

6.69 

Other  Comments 

71 

9.89 

As  previously  stated,  the  full  value  of  Step  10  can  only  be  extracted  through  an  examination 
of  the  res^nses  of  households  to  specific  content  or  questions.  As  such,  it  is  difficult  to 
present  a  general  overview  of  the  written  comments.  However,  apart  from  the  objections  to 
the  disclosure  of  income,  the  main  recurring  themes  for  many  of  the  written  comments  on 
the  NCT  questionnaire  are: 

the  perceived  waste  of  government  money  and  time; 

the  question  is  too  personal  or  too  difficult; 

exasperation  regarding  the  question  purpose  and  data  need; 

difficulty  understanding  the  ethnic  origin  question; 

objection  to  the  ethnic  and/or  race  question; 

frustration  based  on  missing  the  Step  9  skip  instruction; 

difficulty  estimating  the  time  spent  on  various  household  activities; 

difficulty  estimating  the  hours  worked  per  week;  and 

frustration    r^arding   the  perceived    duplication    of  income  reporting    (Revenue 

Canada). 

Appradix  F  reproduces  a  select  number  of  the  written  comments  in  order  to  provide  a 
flavour  for  the  concerns  of  various  households.  These  comments  have  been  extracted  to 
roughly  reflect  the  most  fiequent  NCT  questions  mentioned  in  Stq»  10. 


4.  The  Message  for  Coisus  Tsikeis 

There  are  two  types  of  messages  for  Statistics  Canada  conveyed  through  households'  written 
commits  about  the  NCT.  The  first  message  concerns  the  identification  of  problematic 
questions  (e.g.  design,  wording,  concepts).  As  previously  mentioned,  this  message  is  directed 
at  subject  matter  areas  for  consideration  and  at  the  collection  and  communications  projects 
for  preparation. 

The  second  message  gleaned  from  an  analysis  and  examination  of  these  written  comments 
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is  larger,  albeit  less  tangible.  That  is,  the  Canadian  political  cultural  is  becoming  more 
combative  with  less  consensus  on  any  given  issue  and,  as  such,  Canadians  are  less  likely  to 
view  government  as  neutral  and  acting  in  the  national  interest.  Although  Canadians  continue 
to  provide  the  requested  information,  it  appears  to  be  critical  that  respondents  understand 
the  reason  this  information  is  collected  as  a  prerequisite  for  their  cooperation. 

As  stated  on  the  title  page  of  STC  publications,  Canada  owes  the  success  of  its  statistical 
system  to  a  long-standing  cooperation  between  Statistics  Canada  and  the  citizens  of  Canada. 
This  cooperation  cannot  be  taken  for  granted.  Statistics  Canada  appears  to  be  approaching 
what  can  be  considered  the  acceptable  limit  of  census  taking  in  terms  of  length  and  difficulty 
of  the  census  questionnaire. 
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APPENDIX  A 
NCT  Stqp  10  Questions 


49.     Who  completed  this  questionnaire? 

■ 

Mark  as  many  circles  as  applicable. 


1  O  Person  1 

2  O  Person  2 

3  O  Person  3 

4  O  Person  4 

5  O  Person  5 

6  O  Person  6 

7  O  A  person  who  is  Gsted  on  another  questionnaire  for  this  dwelling 

8  O  A  person  who  does  not  live  here 


)TEP    10 


You  have  now  answered  all  the  census  questions. 

Would  you  please  answer  a  few  questions  regarding  this  census  test  questionnaire. 


Did  you  find  any  of  the  steps  on  pages  2  and  3 
difficult  to  answer? 

1  O  No   —  Go  to  Question  B 

2  O  Yes  —  Write  the  step  number  and  -give  the  reason  for 

thedlfficuHy 


Step  Na 


Reason 


ige  30 


B.    Did  you  find  any  of  the  test  questions  difficult? 

3  O  No   —  Go  to  Question  0 

4  O  Yes  —  Write  the  question  number  and  give  the  reason 

for  the  difficulty. 


Question  No. 


Reason 
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C.    Did  you  use  the  Guide? 

5  O  No    —  Go  to  Question  E 

6  O  Yes  —  For  which  steps  or  questions? 


step  No. 

□ 
□ 


Question  No. 


□ 
□ 
□ 


E.    Did  you  use  the  Census  Help  Line? 

1  O  No 

2  O  Yes  —  Was  it  helpful? 

3  O  Yes 

4  O  No 


D.    Was  the  Guide  helpful? 
7   O  Yes  —  Go  to  Question  E 

fl  O  No  —  Write  the  steps  or  questions  for  which  the  Guide 
was  not  helpful. 


Step  No. 


Explain 


F.    Are  there  any  test  questions  to  which  you  have 
objections? 

5  O  No    —Go  to  step  11 

6  O  Yes  —  Write  the  question  numtier  and  give  the  reason 

for  your  objection. 


Question  No. 


Reason 


Question  No. 


Explain 


□ 


STEP   11 


You  have  now  completed  your 
test  questionnaire. 

Please  mail  it  today  using  the 
enclosed  postage-paid 
envelope. 

Thank  you  for  your  cooperation. 
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APPENDIX  B 
Missiiig  Values  and  Inconsistent  Responses 


11     Question  /Response 

LPS 

Spedal      1 

Households  (n) 

12,273 

2,839 

Question  A  -  Yes  &  No 

2 

0 

Question  A  -  missing  values  [1] 

2,251 

694 

Question  B  -  Yes  &  No 

4 

0 

Question  B  -  missing  values 

2,789 

790 

Question  C  -  Yes  &  No 

4 

1 

Question  C  -  missing  values 

2,355 

716 

Question  D  -  Yes  &  No 

11 

1 

Question  D  -  missing  values  [2] 

n/a 

n/a 

Question  C  -  No  &  Question  D  -  Yes 

677 

184 

Question  C  -  No  &  Question  D  -  No 

928 

174 

Question  E  -  Yes  &  No  [3] 

2 

0 

Question  E  -  Yes  «&  No  [4] 

2 

1 

Question  E  -  missing  values 

2,408 

749 

Question  E  -  No  /  Was  it  helpful  -  Yes 

7 

3 

Question  E  -  No  /  Was  it  helpful  -  No 

15 

2 

Question  F  -  Yes  &  No 

2 

0 

1    Question  F  -  missing  values 

3,361 

897 

[1]     Missing  values  calculated  residually.  For  Question  A  (LFS): 

2,251  =  12,273  -  (9,732  yes  +  290  no). 
[2]     It  is  not  possible  to  differentiate  between  households  which  did  not  answer  Question 

D  (i.e.  missing  values)  and  households  for  which  Question  D  did  not  apply  because 

of  the  skip  pattern  in  Question  C. 
[3]     Did  you  use  the  Census  Help  Line? 
[4]     Was  it  helpful? 
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APPENDIX  C 
Frequaicies  of  Breakdown  Variables 


^^S^i^^^^^  ^ 

LFS 

Spedal          1 

Households  (n) 

12,273 

2,839 

Regional  Office  [1] 

(2) 

(0) 

St.  John's 

634 

n/a 

Halifax 

2,067 

249 

Montreal 

2,298 

721 

1        Sturgeon  Falls 

1,052 

n/a 

Toronto 

2,322 

455 

Edmonton 

2,789 

1,135 

Vancouver 

1,109 

279 

1    Formtype 

(139) 

(30) 

English 

9,728 

2,236 

Frwich 

2,406 

573 

Completed  Questionnaire   [2] 

(285) 

(70) 

Person  1 

5,902 

1,530 

Person  2..n 

4,831 

830 

1        Other  Person 

1,255 

409 

Household  Size  [3] 

1  person 

2.671 

642 

2-6  persons 

9,522 

2,157 

1         >  6  persons 

80 

40 



[1]     Values  in  parenthesis  indicate  missing  values. 

[2]     NCT  Question  49;  "other"  indicates  person  not  in  household. 

[3]    HHLDSIZE  a  derived  variable;  as  such,  no  missing  values. 
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APPENDIX  D 
Statistical  Tests  of  j^glish  and  French  Responses 


Using  a  chi  square  distribution,  observed  differences  in  household  responses  by  language  of 
questionnaire  were  tested,  in  two  particular  cases,  to  determine  if  the  results  were 
statistically  significant.  In  both  cases,  the  LFS  and  Special  sample  components  were 
combined  for  the  test. 


Question  D  -  Was  the  Guide  hdpM? 

Hq  Responses  to  the  helpfulness  of  the  Guide  are  independent  of  the  language  in  which 
the  NCT  questionnaire  was  completed  (i.e.  there  is  no  statistically  significant 
difference  between  English  and  French  responses  to  the  question  on  the  helpfulness 
of  the  Guide). 

chi  square    933.85   >  6.64  (a  =  .01) 

Therefore,  reject  the  null  hypothesis  and  conclude  that  the  Guide's  helpfulness  is  not 
independent  of  the  language  in  which  the  responding  household  chose  to  complete  the  NCT 
questionnaire.  The  observed  difference  in  responses  would  occur  by  chance  only  once  if  this 
test  was  repeated  one  hundred  times. 


Question  E  -  Was  the  CHL  hdpfiil? 

Hq  Responses  to  the  helpfulness  of  the  Census  Help  line  are  independent  of  the 
language  in  which  the  NCT  questionnaire  was  completed  (i.e.  there  is  no  statistically 
significant  difference  between  English  and  French  responses  to  the  question  on  the 
helpfulness  of  the  Crasus  Help  line). 

chi  square    3.91   >  3.84  (a  =  .05) 

Therefore,  reject  the  null  hypothesis  and  conclude  that  the  Census  Help  Line's  helpfulness 
is  not  independent  of  the  language  in  which  the  responding  household  chose  to  complete 
the  NCT  questionnaire.  Presumably,  this  language  would  also  be  the  one  which  the 
household  would  use  in  phoning  the  Census  Help  Line.  The  observed  difference  in  responses 
would  occur  by  chance  only  once  if  this  test  was  repeated  twenty  times. 
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APPENDIX  E 
Written  Comment  Coding/Clasafication 


Position  Field 

1-2  Sampling  Frame  Listing  Number  {1,..258} 

3-4  Regional  Office 

11  St.  John's 

12  HaUfax 

13  Montr&l 

14  Sturgeon  Falls 

15  Toronto 

17  Edmonton 

18  Vancouver 
5-6                 Sample  Componait 

01  Blacks  in  Halifax 

02  Asians  in  Montr^ 

03  Blacks  in  Montreal 

04  Latins  in  Montr&d 

05  Asians  in  Toronto 

06  Blacks  in  Toronto 

07  Aboriginals  in  Winnipeg 

08  M6tis  in  Winnipeg 

09  Aboriginals  in  Regina 

10  M^tis  in  Saskatoon 

11  Aboriginals  in  Edmonton 

12  Asians  in  Vancouver 
99LFS 

7-8  Step  10  Question    {A,B,D,F,X}, 

where  X  denotes  back  cover 

9-10  if  A  or  D  (step)  -  NCT  step  {sl,..s9} 

if  B,  D  (question)  or  F        -*  NCT  question  {1,..49} 

if  X  -*  NCT  question  {l,..49}or  XX  if  not  question  specific 

11  Comment  Classification 

1  Response  Burden  (length/difficulty  or  relevancy) 

2  Privacy  Concerns  (disclosure  or  necessity) 

3  Other  Objection 

4  Collection/Content  (constructive  criticism) 

5  Other  Comments 

12-80  open  field  to  note  any  germane  characteristics  of  respondent/household 
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APPENDIX  F 
Selective  Wiittea  Commoits 


General: 

•  "The  questionnaire  should  contain  instructions  on  how  to  complete." 

•  "Manque  d'exemples  et  de  clarte" 


• 


• 


"We  feel  that  the  Government  of  Canada  would  better  serve  its  citizens  by  creating 
a  nation  of  Canadians,  not  a  federally  funded,  special  interest  group  society." 

"Statistics  Canada  should  help  senior  citizens  with  these  questionnaires.  Some 
questions  are  puzzling  and  eyesight  is  a  problem.  Don't  want  to  be  phoning  at  my  age 
and  if  government  offices  are  busy,  one  would  have  to  keep  trying  to  get  through. 
Some  help  from  ag«it  delivering  test  would  shorten  this  test  considerably." 

"Toutes  les  questions  poshes  dans  ce  questionnaire,  ^  I'exception  des  langues  parlees, 
Jl  mon  avis,  sont  une  perte  de  temps  car  I'information  peut-etre  obtenue  des  rapports 
d'imp6t  done  une  duplication  de  compilation  d' information... Le  questionnaire  ne 
devrait  pas  contenir,  ni  I'adresse,  ni  le  nom,  mais  seulement  un  numdro  qui  pourrait 
^tre,  si  n6cessaire,  compar6  avec  le  questionnaire... il  y  a  ttop  d'information  qui  est 
confidentielle     et  de  ce  fait  ne  devrait  pas  figurer,  ni  nom,  ni  adresse... 

"The  whole  tone  of  the  questionnaire  accentuates  Canada's  differences  -  ethnic, 
cultural,  linguistic.  It  is  time  that  an  effort  was  made,  particularly  by  government 
agencies,  to  minimize  the  differences  and  promote  Canada  as  a  nation." 

"Respondent  apologized  for  not  completing  form  but  said  she  had  op^ied  it  and 
found  it  looked  intimidating  so  didn't  take  the  time  to  complete  it"  (via  Interviewer) 


Privacy 

•  "Trop  de  questions,  il  est  suffisant  de  recueillir  le  nom,  la  date  de  naissance,  I'adresse 
et  le  revenu" 

•  "I  believe  that  it  should  not  be  mandatory  to  fill  out  a  survey  such  as  this.  I  find  it 
to  be  invasion  of  privacy  and  a  waste  of  my  valuable  time." 

•  "Ing^rance  dans  ma  vie  priv6e  qu'on  a  pas  le  droit  de  refuser" 
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•  "Questions  #26  thought  #48  inclusive  were  unnecessary  and  frankly  none  of  Statistics 
Canada's  business.  A  national  census  test  should  focus  only  on  the  number  of 
Canadian  citizens,  their  education  and/or  training.  Their  age,  sex,  origin,  address  etc. 
Over  50%  of  this  test  was  in  my  opinion  asking  personal/confidential  questions. 
Income  statistics  should  be  obtained  from  Taxation  Canada  when  and  if  it  is 
necessary" 

•  "Questions  26  to  46.  This  information  is  nobody's  business.  I  was  waiting  for:  What 
kind  of  toilet  paper  this  person  uses?  Q  1  ply  Q  2  ply" 

•  "First  time  she  saw  such  a  detailed  questionnaire.   It  made  her  feel  apprehensive 
about   completing   it,  and  about  providing  so  much  personal    and  private  detail. 
However,  once  I  explained  the  background  purpose  of  the  test,  she  became  very 
comfortable  responding,  except  for  question  18. "(via  Interviewer) 

Step  9: 

•  "...r^ponsepour  enfant  en  1991?" 

•  "Person  6  is  an  infant  so  they  are  non-applicable" 

•  "aucun  rapport" 

•  "No  box  for  those  retired,  should  proceed  to  Question  46  bypassing  Question  29-33 
(irrelevant)." 

Question  2: 

•  "Did  not  know  if  I  was  suppose  to  check  circle  under  Person  1  - 1  am  related  to 
myself." 

Question  16: 

•  "After  200  years,  I  no  longer  need  to  identify  my  origins." 

•  "Does  this  mean  parents,  grandparents,   ...?" 

•  "What  difference  does  my  ancestry  make?" 

•  "Ancles  ^  quel  degr6?" 

•  "Ne  sp&ifie  pas  ce  qu'est  un  ancetre" 
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•  "How  many  generations  to  go  back  -  guide  not  clear." 

Question  18: 

•  "lam  angered  and  offended  by  this  question." 

•  "What  is  the  purpose  of  this  question?" 

Question  26/27 

"I  do  not  keep  track  of  the  hours  spent  on  household  activities." 

"Ne  savait  pas  ce  que  cela  voulait  dire" 

"^valuer  le  nombre  d'heures?" 

"What  we  do  at  home  is  personal." 

"Travaux  menagers  pour  moi?  (mSme  si  j'habite  avec  une  autre  personne);  oti?  chez 
moi?  chez  d'autres  personnes?" 

"Question  ambigue" 

"I  though  caring  for  your  children  was  a  responsibility  of  parenting." 

"My  family  volunteers  a  lot  but  at  certain  times  during  the  year." 

Question  42: 

•  "Why  do  you  want  to  know  where  we  work?" 

•  "C'estexag€r6,  c'est  la  Gestapo" 

•  "Ne  connait  pas  I'adresse  exacte" 

Question  46: 

•  "Difficult  to  answer!" 

•  "Question  personnelle" 
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•  "Ces  informations  sont  sur  le  rapport  d'impot  et  c'est  Ik  que  je  me  suis  r6f6r6  pour 
obtenir  rinformation" 

•  "Ne  vois  pas  k  quoi  sert  cette  information  dans  le  but  statistique" 

•  "I  do  not  fiilly  understand  what  you  need  the  information  for  and  I  do  not  folly 
understand   the  questions." 

•  "confidentiality" 

•  "Ca  m'a  agac6.  Je  trouve  5a  personnel  et  c'est  agressant  de  faire  entrer  ainsi  de  ses 
choses  personnelles  par  un  questionnaire  g6r6  par  des  Strangers.  Je  trouve  que  c'est 
une  question  qui  d^passe  le  mandat  d'un  recensement". 

•  "If  people  fill  out  income  tax  returns  they  should  not  be  required  to  locate  their 
records  for  the  Census." 

•  "I don't  mind  filling  out  a  questionnaire  but  Statistics  Canada  must  have  access  to  our 
income  tax  - 1  don't  have  time  to  deal  with  too  many  specifics  details  re;  postal  codes 
etc." 
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